A generic architecture and dialogue model for multimodal interaction

This paper presents a generic architecture and a dialogue model for multimodal interaction. Architecture and model are transparent and have been used for different task domains. In this paper the emphasis is on their use for the navigation task in a virtual environment. The dialogue model is based on the information state approach and the recognition of dialogue acts. We explain how pairs of backward and forward looking tags and the preference rules of the dialogue act determiner together determine the structure of the dialogues that can be handled by the system. The system action selection mechanism and the problem of reference resolution are discussed in detail

A System for Monitoring Stroke Patients in a Home Environment

Currently, the changes of functional capacity and performance of stroke patients after returning home from a rehabilitation hospital is unknown for a physician, having no objective information about the intensity and quality of a patient's daily-life activities. Therefore, there is a need to develop and validate an unobtrusive and modular system for objectively monitoring the stroke patient's upper and lower extremity motor function in daily-life activities and in home training. This is the main goal of the European FP7 project named “INTERACTION‿. A complete sensing system is developed, whereby Inertial Measurement Units (IMU), Knitted Piezoresistive Fabric (KPF) goniometers, KPF strain sensors, EMG electrodes and force sensors are integrated into a modular sensor suit designed for stroke patients. In this paper, we describe the systems architecture. Data from the sensors are captured wirelessly and stored in a remote secure database for later access and processing via portal technology. In collaboration with clinicians and engineers, clinical outcome measures were defined and the question of how to present the data on the web portal was addressed. The first implementation of the complete system includes a basic version of all components and is currently being extended to include all sensors within the INTERACTION system